Skip to content

fix(config): refuse startup that would orphan the KB registry; probe both ends for zero vectors#36

Merged
lfnothias merged 2 commits into
mainfrom
fix/session-db-startup-guard
Jul 10, 2026
Merged

fix(config): refuse startup that would orphan the KB registry; probe both ends for zero vectors#36
lfnothias merged 2 commits into
mainfrom
fix/session-db-startup-guard

Conversation

@lfnothias

Copy link
Copy Markdown
Collaborator

Two defects surfaced by an adversarial audit of main after PRs #32#35 merged. Each is a follow-up to code those PRs introduced.

1. Startup can silently orphan an existing KB registry (blocking)

PR #33 made the session/KB-metadata SQLite path resolve from config.database.path. Before that, every entry point hardcoded ./data/perspicacite.db and ignored the config key. The shipped config.example.yml — the documented starting point (cp config.example.yml config.yml) — sets database.path to ~/.local/share/perspicacite/memory.db. So a deployment created before #33 keeps its knowledge bases in the legacy working-directory file, while its config names a home-directory path that does not exist. After #33, honouring the config opens an empty database and list_kbs() returns an empty list: every knowledge base vanishes from the UI, and new metadata is written where the vectors do not follow.

#33's guard only defended the case where database.path is unset; the documented config sets it explicitly, so the guard did not fire for exactly the mainstream upgrade path.

guard_session_db_path now refuses to start when the configured database is absent and the legacy ./data/perspicacite.db still holds a non-empty kb_metadata table. The error names both paths and the fix. No data is moved — relocating a production registry implicitly is riskier than a loud stop. PERSPICACITE_ALLOW_NEW_DB=1 opts into a deliberately fresh instance. Wired into the web app, the MCP server and the two CLI entry points.

Verified against the real production registry and config: the guard raises with the correct message and paths, and the escape hatch bypasses it.

2. The embedding-health probe missed tail poisoning (advisory)

PR #32 added embedding_health to GET /api/kb/{name}/stats, sampling a bounded prefix (first 128 chunks). A mid-ingest embedding outage leaves a contiguous run of zero vectors, usually at the end of a collection, which a prefix-only sample reports as healthy. The probe now reads both ends, and the block gained total_chunks and complete so a clean sample of a large collection is not mistaken for a proven-clean collection.

Tests

  • test_config_database_path.py: guard raises on the orphaning scenario; the escape hatch, an absent legacy DB, an empty legacy DB, and a config pinned at the legacy path all pass through.
  • test_embedding_health_probe.py: zeros beyond the first 128 chunks flip degraded to true; a clean large sample is reported complete: false.

CI's unit suite: 2433 passed, 5 skipped, 0 failed. ruff clean on the new module; mypy clean on config/paths.py.

@lfnothias lfnothias merged commit 547cfac into main Jul 10, 2026
3 checks passed
@lfnothias lfnothias deleted the fix/session-db-startup-guard branch July 10, 2026 16:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant